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FY-75 KEP PERFORMANCE REPORT 


rr "B 1976 


I . INTRODUCTION 


This report presents an analysis of evaluations of the 
IC's performance with respect to the FY-1975 Key Intelligence 
Questions (KIQs) . 1 It examines evaluations of Community 
performance against each KIQ, the scope and cost of Community 
collection and production efforts related to addressing 
KIQs, and assesses the Key Intelligence Question Evaluation 
Process (KEP) methodology which evaluates Community performance. 

The Key Intelligence Questions are issued annually as 
guidance to the Intelligence Community for the collection 
and production of intelligence on subjects of major im- 
portance to national-level consumers during the ensuing 
year. Key Intelligence Questions do not address the full 
spectrum of consumer needs. Additional important categories 
of intelligence collection and production not included in this 
priority national intelligence guidance encompasses: lower 
priority national intelligence subjects, and departmental and 
tactical intelligence. 

KIQs are intended to insure that the subjects are given 
priority in the regular collection and production activities 
of appropriate elements of the IC; to enable review of 
collection and production planning by the member agencies 
of the IC; to provide the basis for a recapitulation and 
evaluation, at the close of the FY, of the performance 
of the IC — and individual agencies — on each KIQ; with 
the objective that conclusions drawn from such recapitulations 
and evaluations be used as factors in Community resource 
allocations. 

In developing the FY-75 KIQs, NIOs were asked to bear 
in mind the DCI's FY-75 Substantive Objectives for the Community 
and to consider DIA's Critical Near-Term Defense Intelligence 
Objectives as a basic input for military-related KIQs. 2 
These instructions also reflected the DCI's desire that the 
number of KIQs be limited to focus clearly upon a single 
subject. 


Annex A lists FY-75 KIQs, and DCI's FY-75 Substantive 
Objectives to which they pertain . 

Memo dated 5/29/74, Subject : The KIQ-KEP Program for 

FY 1975. 


Approved For Release 2004/1 


80M00636A0001 00020001 -8 



1 , 



Approved 


For Release 20' 



DP80M00636A0001 00020001 -8 


Recognizing that the KIQ is designed to communicate current 
priority needs, there was a tnedency to include as many aspects 
of the worldwide intelligence problem in the KIQ list as 
possible. While more subjects can be included in each KIQ by 
developing more generalized and inclusive statements, this 
tends to make the list less exclusive and widens the gap 
between the implicit relative importance of the KIQs themselves. 
Since KIQs are to communicate priority needs for tasking and 
evaluation of performance, the number of KIQs must be controlled. 
Each must be meaningfully selective (and unambiguous) with 
respect to geography and topic. Because KIQs tend toward the 
General, and because of the conceptual absence of judgments on 
the relative values of individual KIQs, it is essential that 
clear, precise collection and production objectives be set 
forth as the basis for development of Community collection and 
production plans (strategies) . 

The Key Intelligence Question Evaluation Process (KEP) 
was designed to be a simple, iterative process to evaluate 
Community performance. Structurally uncomplicated, yet 
containing enough data to support a meaningful evaluation 
of Community performance, the KEP provides a substantive 
foundation for USIB/IRAC discussions concerning the utilization 
of NFIP resources. In particular, KEP results are intended 
to improve the process by which USIB/IRAC principals become 
informed on resource matters by providing information on 
the relative effectiveness and cost of Community performance 
in answering the KIQs. Thus, the KEP informs program managers 
about resource expenditures on KIQs, rather than providing 
a vehicle for making resource allocation decisions. Its 
design recognizes the complex processes that enter into 
formulation of individual NFI programs, and the difficulty 
in devising resource allocation procedures. 

To have a positive impact on resource deliberations, 
the KEP must, to some extent, be an audit process which 
enables the summing of KIQ and non-KIQ efforts by program. 
Presently, the KEP consists of a review of the status of 
knowledge on each KIQ and the formulation of collection and 
production objectives; the development of a Community collec- 
tion and production strategy to answer the question; the 
assessment of Community performance of the tasks reflected 
in individual KIQ strategies; and identification of the 
costs associated with KIQ activities at the KIQ level. The 
principal features of the KEP are: 
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• The formulation of collection and production 
and the development of Community collection and 
production strategies for each KIQ; 

• the acceptance of responsibility by individual 
agencies to collect or produce on individual 
KIQs; 

• the linking of the resources expended to the 
information gained; and 

• the assessment of consumer satisfaction with 
the intelligence produced. 

In developing KEP procedures it was necessary to strike 
a balance between being overly precise, and being too general 
in acquiring cost data. At one extreme, too much precision 
would be misleading in light of the credibility of obtainable 
data. At the other extreme, overly general and unsystematized 
cost data would not be helpful as it would inhibit comparing 
and contrasting Community performance among KIQs and NFI 
programs . 

In defining the KEP, two general approaches were con- 
sidered to acquire necessary cost data. One was a unique, 
highly-structured accounting methodology providing precise 
data that would have a heavy management burden and which, 
in all probability, would produce data far too precise for 
KEP's intended purpose. The other, based on CIRIS, was a 
less precisely structured methodology — which, while it will 
never be suitable for direct use in resource allocation 
decisions — provides cost data which could be used in Community 
resource deliberations. The second, more generalized approach, 
was selected because: 

• KIQs do not encompass the full range of 
Community activities; 

• KEP is an information mechanism, not a 
resources allocation system; 

• fewer management resources would be required ; 
and 

• recently changed procedures for obtaining CIRIS* 
data make credible resource data readily available. 


* Consolidated Intelligence Resources Information System — 
operated and managed by the ICS to identify and display the 
expected distribution of intelligence resources in official 
programs and budgets . 
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The FY-75 KEP Program began on 12 August 1974 with the 
circulation of NSCIC-approved Key Intelligence Questions for 
Fiscal Year 1975. Guidance and Instruction for Performance 
Reporting 3 and a Listing of FY-75 KIQ Objectives jointly 
developed by the NIOs and ICS were distributed to the IC on 
9 and 30 June 1975, respectively. 4 

On 1 July the FY-75 KIQ performance period ended; per- 
formance assessments and collection and production performance 
data required by performance reporting instruction were requested 
by 1 August. When the data acquisition phase of the evaluation 
program closed on 5 November, Performance Assessments Reports 
on 62 of the 69 FY-75 KIOs had been received, the last on 


In assessing performance, NIOs evaluated Community and 
individual agency performance on each KIQ (and on the 
collection/production objectives identified in KIQ Strategies) . 
The relative contribution each collection method made to 
the information gain on the KIQ was also assessed. 

Production elements of the Community — CIA (DDI and 
DDS&T) , State (INR) , and Defense (DIA) — provided the full 
range of production performance data required by the KEP 
(production organizations, products, costs, and problems), 
through a series of incremental inputs, the last of which was 
received on 23 October from DIA. 


Collection performance data was obtained by the appro- 
priate USIB collection committees — SIGINT, COMIREX and Human 
Resources — from IC program managers. In content, this data 
paralleled that provided by producers, identifying collectors 
and collection efforts, costs and collection problems. 

The final input of this data — information on processing costs 
was received from CIA/OEL on 23 October. 


3 USIB/IRAC-D-22 .1/38 dated 9 June 1975. 

4 Annex B, USIB/IRAC-D-22 . 1/40 dated 30 June 1975, lists 
KIQ objectives identified in FY-75 KIQ strategies. 
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necessitating the even more difficult task of postulating 
Soviet perceptions of their options and success expectations — 
the Community needs to know what is required with enough 
specificity to enable program managers to make meaningful 
commitments with respect to the objectives and tasks which 
should be attempted if the KIQ is to be answered. It is not 
sufficient for a strategy to indicate merely that producers 
will produce, and collectors will collect in accordance with 
their capabilities and extant tasking. Without specificity 
and precision at the beginning of the year, the ability to 
identify accomplishments and establish a creditable relation- 
ship among outcome, performance, priorities and resources 
at the end of a performance period will be constrained to 
the precision and explicitness of the least precise strategy. 

On the whole NIO assessments of IC performance on FY-75 
KIQs Were carefully drawn. Individual assessments show con- 
siderable variation in completeness. In some instances 
narratives are inconsistent with quantitative judgments set 
forth in assessment annexes. In many cases, assessments make 
little, or no reference to the collection and production 
strategies which initiated the performance being assessed. 
Performance reports in some cases did not address collection 
and production objectives cited in the Strategies. 

Performance reports on KIQs relating to Soviet policy and 
intentions (Substantive Objective I); and Western Europe (Sub- 
stantive Objective V) do not address IC performance and collector 
contributions with respect to the specific production and 
collection objectives set forth in pertinent strategy reports. 
Assessments of Community performance on Latin American KIQs 
(also Substantive Objective IV), while carefully drawn, use 
language that makes it difficult to determine differences in 
Community achievements on the six KIQs involved. These in- 
consistencies and discrepancies inhibit comparison of performance 
among KIQs with collectors/producers, and makes more difficult 
the task of relating outcome and performance to resource 
expenditures. 

• Community production agencies provided detailed 
production performance data for full range of information 
required by performance reporting instructions. Despite the 
detailed nature of the production performance data provided 
by IC production authorities, it has been extremely difficult — 
almost impossible — to measure Community production performance 
given the differences in approach, precision of expression 
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and level of detail in production strategies; NIO focus on 
interagency production (in some instances virtually ignoring 
departmental/agency production) ; and the inherent difficulty 
of identifying and quantifying KIQ-related current intelligence 
efforts. These difficulties, compounded in inconsistencies 
and ambiguities in reporting KIQ-associated activities and 
products pertinent to more than a single KIQ, could not be 
overcome. Analysis of FY-75 KIQ performance, therefore, does 
not explicitly consider individual departmental or agency 
products in assessing KIQ achievements. 

• Among collectors, KEP collection data requirements 
had the greatest impact on the CCP and NRP because of the 
magnitude of the resources involved, and the universality 
of their application to KIQs. It is instructive to note the 
problem as seen by NSA. — 

"...KEP instructions posed three distinct 
categories of problems : 

a. We were asked to measure the unmeasurable — - 
KIQs deal with the subject content of products; our 
program costs are oriented toward collection targets 
which can contribute directly, or indirectly by 
interpretation to a wide variety of KIQ subjects. 

b. We were asked to cost KIQs at the CIRIS 
Reporting Entity level- -the type of information 
sought by station is almost entirely judgmental 
and the background required to render the judg- 
ments rests with the sub-elements of NSA's management 
structure . 


c. Workload — which warranted computer 

application. 

The following assumptions were the basis for KIQ 
costing : 
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ERRATA SHEET FOR SUMMARY REPORT. 
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